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Introduction 

Bayesian  estimation  of  missile  system  reliability  requires  a  prior  distribution  to  be 
specified  for  a  parameter  of  interest,  such  as  the  system  failure  rate  or  the  system 
reliability  for  example.  However,  many  current  parameter  estimates  available  to 
the  missile  community  are  in  the  form  of  point  estimates  or  design  values  without 
uncertainty  assessments  such  as  the  MIL-HDBK-217  reliability  predictions  (MIL- 
HDBK-217FN2,  February  1995).  This  lack  of  error  bounds  makes  it  problematic  to 
utilize  the  point  estimates  as  priors  for  Bayesian  reliability  assessment.  In  this 
paper,  we  propose  using  the  Maximum  Entropy  Principle  (MEP)  to  convert  point 
estimates  or  design  values,  as  obtained  proforma  from  external  sources,  to  prior 
probability  distributions  convenient  for  Bayesian  analysis.  This  approach  utilizes 
the  physical  concept  of  entropy  to  search  for  the  appropriate  distribution  that 
maximizes  this  entropy,  which  characterizes  the  most  uncertain  prior  distribution 
with  the  available  point  estimates  as  constraints  (e.g.  Bayesian  Inference  and 
Maximum  Entropy  Methods  in  Science  and  Engineering,  July  2007).  We  illustrate 
the  utility  of  the  above  priors  by  applying  them  to  a  Bayesian  reliability  model  of  a 
missile  system. 

Maximum  Entropy  Density 

The  entropy^  (aka  relative  entropy,  Kullback-Leibler  divergence),  of  a  continuous 


^  The  views  expressed  in  this  paper  are  those  of  the  authors  and  do  not  reflect  the  official  policy  or  position  of  the 
us  Navy,  Department  of  Defense  or  the  US  Government. 

^  This  is  not  the  only  entropy  formulation  in  use.  For  example,  the  Shannon  entropy  —  f  p(6)  lnp(0)  dB  is  also 
another  choice,  but  we  have  found  this  formulation  to  be  most  appropriate  for  our  analysis. 


Page  1 


probability  density  function  p{6)  is  defined  by  the  physicist  E.T.  Jaynes  as  (Berger, 
2010) 


£(p)  = 


where  Po(^)  'S  the  natural  invariant  uninformative  density  in  the  Bayesian  sense 
for  the  class  of  distribution  under  consideration.  Missile  pass/fail  data  are 
assumed  to  belong  to  the  class  of  Bernoulli  distributions  Binomial (^1, 6)  (where 
Binomial(n,  O')  denotes  the  binomial  distribution  with  sample  size  n  and 
probability  6)  with  6  as  the  reliability  parameter.  For  this  class  of  distribution,  the 
natural  parameterization  invariant  uninformative  prior  po  is  given  by  the  Jeffreys' 
prior 


Without  any  additional  constraints,  we  note  that  po  is  the  pdf  that  maximizes  the 
entropy.  Suppose  in  addition,  we  have  some  information  about  the  moments  of 
p  (such  as  the  mean,  variance,  quantiles,  etc.),  it  is  desired  to  look  for  a  pdf  that 
maximizes  the  entropy  subject  to  these  constraints.  In  other  words,  we  are 
looking  for  the  most  uncertain  pdf  that  is  consistent  with  the  given  prior 
information.  Using  variational  calculus,  this  density  can  be  written  as  (Berger, 
2010) 


0.(6') 


(1) 


where  the  parameters  roj  are  determined  by  the  known  information 


and  gi(0)  are  predetermined  functions  of  the  parameter  6.  To  illustrate,  let  us 
suppose  that  only  the  mean  is  known  (i.e.,  mean  reliability  p).  Then  we  have  a 
single  constraint  Pi(0)  =  0,  and 


EP{e)  =  p. 
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Claim:  The  maximum  entropy  probability  density  with  mean  [i  is  given  by 


m  = 


/o  (f )  71^0(1  -  0) 


) 


where /„(x)  is  the  modified  Bessel  function  of  the  first  kind  of  order  n. 


To  prove  this,  we  only  need  to  show  that 

1 

f  /6l)\ 

j  po(e)e"‘'de  =  £2 

0 

From  (Abramowitz  &  Stegun,  1972, 13.2.1),  the  following  fact  holds 

1 

I  po(e)  e""  de  = 

0 

where  M(a,b,z)  is  the  Kummer's  function  (Abramowitz  &  Stegun,  1972, 13.1.2). 
Finally,  from  (Abramowitz  &  Stegun,  1972, 13.6.3) 


QED. 

As  mentioned  before  in  (1),  the  parameter  o)  is  determined  by  solving  the 
equation 


1 

j  9p{9')  do  =  p  . 
0 

In  fact,  this  is  the  same  as  solving  the  equation 


(2) 
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which  can  be  done  using  numerical  techniques  such  as  Newton's  method.  Above 
formula  follows  immediately  from  the  fact  that  lo'ix)  = /i(x)  (Abramowitz  & 
Stegun,  1972,  9.6.27)  and 


1 

j  dO 

0 


euj0 


Example.  Suppose  a  missile  point  reliability  estimate  has  the  value  .92.  Then 
equation  (2)  gives  o)  =  6.96. 

Some  representative  max  entropy  pdfs  are  plotted  below. 


Max  Ent  PDF,  0  =  0 


Max  Ent  PDF,  k)  =  10 
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Max  Ent  PDF.  co=  100 


Figure  1.  Examples  of  maximum  entropy  priors  with  different  values  of  o). 
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Missile  System  Reliability  Model 

Missile  system  test  data  are  usually  provided  as  t^),  (^2, 12), ... , 
where  each  Xj  is  Bernoulli  random  variables  with  values  0  (fail)  and  1  (success) 
and  tj  is  the  time  that  the  data  are  collected.  Our  goal  is  to  estimate  the 
reliability  6  using  this  data  set.  Recall  that  reliability  can  be  dependent  on  other 
factors  (e.g.,  time)  and  the  method  used  to  specify  such  dependence  is  to 
construct  a  reliability  regression  model  (Hamada,  Wilson,  Reese,  &  Martz,  2010). 
For  simplicity's  sake,  we  will  only  consider  time  as  the  dependent  variable  and 
construct  an  intercept-exponential  model  for  reliability, 

R(t)  =  6{t)  =  , 

where  ao  and  %  are  the  unknown  parameters  to  be  estimated  (the  notations  R 
and  6  are  interchangeable).  For  this  to  be  a  valid  reliability  model,  there  is  an 
implicit  assumption  that  ao  +  a^t  <  0  at  all  times,  so  that  R{t)  <  1.  This 
condition  is  satisfied  whenever  Uq  (the  intercept,  which  provides  an  initial 
reliability  estimate  e“o  at  t  =  0  )  and  (the  rate  of  failure)  are  both  less  than  or 
equal  to  0.  The  computed  likelihood  function  becomes 

m 

L(data|  ao,ai)  = 

i=l 

Note  that  missile  pass/fail  data  are  usually  left  and  right  censored  data,  since  a 
failed  test  usually  indicate  the  failure  occurred  sometime  in  the  past,  while  a 
successful  test  means  the  system  is  still  good  for  some  time  after  the  test. 
Therefore,  this  is  the  appropriate  form  of  the  likelihood  equation  given  censored 
data  (National  Institute  of  Standards,  2013).  We  now  describe  some  approaches 
on  how  the  max  entropy  distributions  can  be  used  as  priors  for  the  parameters  Uq 
and  a^.  We  identify  two  main  methods  for  Bayesian  reliability  estimation,  the 
independent  endpoints  method  and  the  independent  decrement  methods  (e.g. 
Lawrence  &  Van  der  Wiel,  2006). 
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Independent  Endpoints  Method 

In  many  cases  dealing  with  missile  reliability,  we  are  provided  with  the  point 
estimate  of  reliability  at  certain  time  points  only.  Suppose  we  know  the  system 
reliabilities  and  R2  at  two  different  times  and  T2  respectively.  In  this  case, 
we  can  use  the  above  results  to  specify  a  max  entropy  distribution  as  prior  with 
reliability  R^  at  time  having  mean  R^  ,  and  another  max  entropy  prior  with 
reliability  R2  at  time  T2  having  mean  R2  .  The  independent  endpoints  method 
allows  one  to  obtain  the  priors  for  aQ  and  %  under  the  assumption  that  the  joint 
density  is  the  product  of  the  two  priors,  i.e. 

where  C  is  some  normalizing  constant.  Since  R^  =  e“o+ai7’i  3^01  ^ 

by  a  standard  change  of  variables,  we  derived  the  prior  joint  density  of  {clq,  %) 
as 


7r(ao,ai)  =  C 


(  a,(T2+T.)\ 

,(ajie“o+ai7’i+(j2e“o+ai7’2)g(^ao+ - 2 - jlT’^  _ 


>  (3) 


and  the  posterior  density  is 


p(ao,  All  data)  a  7r(ao,  ai)L(data|  ao,  %). 

Bayesian  mean  estimates  and  confidence  limit  (credible  intervals)  calculations  for 
the  reliability  model  can  now  be  computed  from  the  above  posterior  distribution, 
i.e. 


0  0 

W)  =  =  II  gOo+ait  p(^aQ,  a^ldata)  duoda^  . 

“00  —00 

While  these  numerical  integrations  can  be  complicated,  these  calculations  are 
now  handled  routinely  using  the  Markov  Chain  Monte  Carlo  (MCMC)  procedure, 
available  for  example  on  popular  statistical  packages  such  as  SAS.  The  following 
figure  illustrates  the  calculated  system  reliability  with  the  above  concepts. 
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Missile  System  Reliability 


requirement 


MONTHS 

mean  —  —  —  95% 


Figure  2.  Missile  system  reliability  from  the  independent  endpoints  method. 
Shown  are  the  mean  reliability  curve  (solid)  and  the  95%  two-sided  credible  limits 
(dash). 


Independent  Decrements  Method 

Another  method  that  has  seen  some  application  is  the  independent  decrement 
method.  Instead  of  having  reliability  priors  specified  at  two  different  times  as 
above,  we  have  the  following  scenario,  a  point  estimate  of  the  reliability  is 
known  at  an  initial  time  T^,  and  a  point  estimate  of  reliability  decrement  A(T2>  7i) 
is  also  known  at  a  later  time  T2,  so  that  R(T2)  —  As  in  the  independent 

endpoints  method,  we  specify  a  max  entropy  distribution  as  prior  with  R^  at  time 
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having  mean  and  specify  another  max  entropy  prior  for  the  reliability 
decrement  (which  we  assume  to  be  expressible  in  the  same  model  form)  with 
A  =  A(r2,  r^)  having  mean  A.  Under  the  assumption  that  the  decrement  is 
independent  of  the  initial  reliability,  the  joint  density  is  given  by 


As  before,  a  standard  change  of  variables  with  R^  =  and  A=  e“i('^2-7’i) 

gives  the  joint  density  of  (tto,  fli)  as 


7r(ao,ai)  =  C 


2  Jit  _ 


\Ti-T2\ 


■\/\  —  —  gCniTz—Tx) 


(4) 


and  the  posterior  density  is 


p(ao,  ttildata)  a  7r(ao,  ai)L(data|  ao,  %). 

As  before,  Bayesian  mean  estimates  can  be  computed  from  the  above  posterior 
distribution,  i.e. 

0  0 

/?(t)  =  A(/?(t))  ~  j  j  p(ao,  a^ldata)  duoda^  . 

—  00  —00 


Failure  rate  as  point  estimate 

A  special  application  of  the  above  methods  worth  mentioning  is  the  use  of  MIL- 
HDBK-217  point  estimates  for  reliability  predictions.  These  estimates  are  not 
reliability  values  in  itself,  but  rather  point  estimates  of  failure  rates.  We  now 
demonstrate  how  to  apply  the  above  methods  to  obtain  the  maximum  entropy 
priors  for  this  case. 


Suppose  we  are  given  a  reliability  point  estimate  Ro^t  t  =  0 ,  and  the  failure  rate 
A  as  a  point  estimate  similar  to  what  is  provided  in  MIL-HDBK-217  data.  We  can 
write 
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R(t)  =  =  Roe^^^. 


Applying  the  independent  endpoints  method  to  this  case,  we  deduce  that  the 
maximum  entropy  prior  is  given  by 

7r(/?o,ai)  = 

=  C(t)e"o^o/?o~2(l  -  -  Roe^^^y2tRQe^^^ 

g(wo«o+"i«oe“iOt 
=  Cit)  , 

VF^Ve““''  -  «0 

for  some  normalizing  factor  C(t)  that  depends  on  t.  ojq,  0)^  are  determined  by 
solving  the  equations 


with  the  understanding  that  the  mean  E (e“o+ait)  a  e£'(ao)+  £'(ai)t  a  is 

valid  (as  first-order  Taylor  approximation).  Note  that  0)^  =  CL)i(t)  is  also  a 
function  of  t  (for  fixed  A.) .  Equivalently,  the  change  of  variables  Rq  =  gives 
the  max  entropy  prior  as 


7r(ao,  %) 


=  7r(ao,  %,  t)  =  C(t) 


g(woe“o)gao  g(wie“o+ait)^^ao+ait 

—  gag+a^t^ 


=  C(t) 


g(woe“o+Wie“o+ait)^^ao 

Vl  —  e“oVe““i^  —  e“o 


To  see  this,  we  just  set  y  =  0,  T2  =  t,  (xi-^  =  (Dq,  0)2  =  coi  in  equation  (3). 


The  posterior  density  is  given  by 

p(ao,  flildata,  t)  oc  7r(ao,  %,  t)L(data  \  aQ,ay, 


and  the  corresponding  Bayesian  mean  reliability  computed  from  the  above 
posterior  distribution  is 
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0  0 


R(t)  =  £(R(t))  = 


gao+ait  t|data)  daQda^. 


Now  given  the  same  prior  information  Ro^t  t  =  0  ,  and  the  failure  rate  A,  let  us 
assume  that  the  initial  reliability  Rq  is  independent  of  the  failure  rate  We  can 
then  apply  the  independent  decrement  method  and  write 

R(t)  =  =  Roe^^^  =  RoAit), 


and  deduce  that  the  maximum  entropy  prior  is 


1  1  g(wie“i'')jgait 

7r(/?o,ai)  =  C(t)p(/?o)7r(ai)  =  -  j?o)~2 


for  some  normalizing  factor  C(t)  that  depends  on  t.  are  determined  by 

solving  the  equations 


1  + 


=  2e 


) 


where  we  make  the  usual  Taylor  approximation  =  e  . 

Equivalently,  the  change  of  variables  Rq  =  gives  the  max  entropy  prior  as 


7r(ao,  %)  =  C(t) 


g(woe“°)gao 

^e“o(l  —  e“o)  —  e“i^) 


To  see  this,  we  just  set  7^  =  0,  T2  =  t,  (xi-^  =  (Dq,  0)2  =  in  equation  (4). 


Notice  that  7r(ao,ai)  =  7r(ao)7r(  %)  ,  with 


^  (^0)  “  ^0 


g(woe“o)gao 

^g“o(l  —  e“o) 


,n(  %)  =  Ci(t) 


^gait(i_gait)' 


for  some  normalizing  factors  Cq  and  Ci(t).  As  noted  previously,  =  CL)i(t)  is  a 
function  of  t  (for  fixed  A).  Let  us  write  n^a^)  =  ?!(%,  t).  The  posterior  density 
satisfies 
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p(ao,  t|data)  =  p(aol  data,  t  =  0)p(ai|data,  t  >0) 

a  7r(ao)L(data,  t  =  0|  Uq)  ?!(%,  t)L (data,  t  >  0|  %). 

In  this  case,  the  corresponding  Bayesian  mean  estimates  has  a  simple  form  given 
by 


0  0 


fi(t)  =  £(R(t))  =  II  gOo+ait  p(afQ,  a^,  t|data)  daQda^ 

—  00  —00 

^0  r  0 

=  I  e“op(aol  data,  t  =  0)  dUg  I  data,  t  >  0)^% 

^  —  00  ^  —  co 


Comparing  the  two  approaches,  we  observe  that  for  the  independent  decrement 
method,  0  <  A(t)  <  1,  so  that  R(t2)  <  Riih)  whenever  t2  >  t^.  This  method 
therefore  guarantees  that  the  mean  reliability  R(t)  is  monotonically  decreasing 
with  time,  which  is  not  necessarily  the  case  with  the  independent  endpoints 
method.  This  makes  the  latter  a  more  attractive  choice  for  reliability  estimation. 


Perfect  intercept  case 

We  would  like  to  mention  that  in  some  cases,  it  is  useful  to  consider  the  special 
model 


R(t)  = 


where  the  intercept  term  ao  is  ignored  (e.g.  system  is  essentially  assumed  to  be 
perfect  at  t  =  0).  With  known  failure  rate  A  as  point  estimate,  the  max  entropy 
prior  for  %  is  just 


7r(  %)  =  ?!(%,  t)  =  C (t) 


) 


with  roi  determined  by 
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1  + 


=  2e  . 


The  posterior  density  is  given  by 

p(ai,  t|data)  oc  n{a^,  t)L(data|  %)  , 

with  mean  reliability 

0 

W)  =  =  /  p{a^,  t|data)  da^. 

—  00 

Concluding  Remarks 


In  the  absence  of  additional  information  regarding  point  estimates  of  reliability 
numbers,  we  illustrate  a  natural  method  for  obtaining  a  reasonable  probability 
distribution  by  employing  the  principle  of  maximum  entropy  (MEP)  to  the  point 
estimates.  MEP-based  distributions  characterize  the  least  informative 
distributions  with  the  available  point  estimates  as  constraints.  These  distributions 
are  then  applied  as  priors  to  a  Bayesian  system  reliability  model  to  obtain  the 
posterior  reliability  distribution,  which  can  be  used  to  assess  missile  reliability  and 
answer  other  important  reliability  related  questions  such  as  missile  availability, 
maintenance  cycle,  logistic  policy,  firing  doctrine,  etc. 
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